Language identification for the automatic grapheme-to-phoneme conversion of foreign words in a German text-to-speech system

The German language is interspersed wi th words, mostly foreign words, which cannot be converted correctly by German graphemeto-phoneme rules. Almost all of these words belong to the English or French language. In order to convert the whole text correctly, it is necessary to identify the language of each word and then to use the corresponding set of grapheme-to-phoneme rules. The problern of language identification is Centrally important for almest every application of automatic text output when its source is a multilingual text database. FURTHER APPLICATIONS The possibility of detecting the language of a written text automatically opens up some central fields of application. The following two may serve as an example: A library of scientific literature contains, e.g. 1 texts written in English 1 German and French. If a blind person likes to get access to the library via a text-to-speech system 1 this system may detect the language and acti va te the corresponding rule set and then read the selected text aloud correctly. Or 1 a list of available articles of a selected topic may serve as an example. Such a list may be available only on micro fiches at the moment but callable by phone if the language of the title and the author's name can be detected automatically. BASIC SYSTEM For all these applications the same basic architecture can be used (figure 1). Supposed that grapheme-to-phoneme rule sets with an accuracy of 100% are available for the three languages mentioned above 1 the performance of a multilingual system only depends on the quality of the language identifier. The main problern is how to detect the language aut:omatically. The only informa·tion that can be used by a language identification system is the character code of the written text 1 the word length and eventually the domain of a word within a sentence. ASCII-text (L.1, L.2, .. , L.n) I languageldentlfler grapheme-to-phoneme converters Fig. 1: Multilingual Grapheme-tc-Phoneme Conversion Basically 1 two methods of evaluating this information in order to identify the language are feasible: 1. Systems which are based on statistics calculate the mean probability of all grapheme transitions of a word for the possible languages and select the most probable one. 2. Rule-based systems evaluate multiple features in order to exclude step by step those languages a word definitely does not belong to. The necessary information for this evaluation can be found either on word or sentence level. USED CORPORA For the development and the evaluation of all language identification systems that will be discussed in the following the same text corpora have been used. These corpora have been collected in the ESPRITProject 291/860. They consist of bureau communication texts 1 200 1 000 words in English 1 French and German each. From these corpora word lists have been extracted and subsequently they have "~een freed from foreign words 1 words containing language specific characters and abbreviations.